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Abstract 

The purpose of this feasibility study was to inform the development of an intervention to support reading and self- 
regulation for students with significant reading difficulties and disabilities (RDs), including dyslexia. Participants were 21 
special educators, dyslexia specialists, and reading interventionists and 48 students in Grades 2 to 4. Student outcomes 
were compared using a quasi-experimental design; some teachers provided the research intervention (Idea Detectives 
[ID] group), whereas others provided the reading interventions typically offered in their schools (business-as-usual [BAU] 
group). Nearly 90% of BAU students received alternate evidence-based interventions. Results showed that student 
outcomes did not differ between the ID and BAU groups. Observational data indicated that revisions were needed to 
improve the intervention’s feasibility, and qualitative teacher data identified barriers to consistent implementation, as 
well as strengths and shortcomings of the intervention. Teacher data suggested strong support for the inclusion of self- 
regulation instruction with reading intervention. This study illustrates the importance of teacher—researcher collaborations 


for the development of instructional interventions. 
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Even when provided with quality early reading interventions, 
many students in Grades 2 to 4 have significant reading dif- 
ficulties and disabilities (RDs). Students with significant 
RDs, including students with dyslexia, require intensive 
reading interventions (Vaughn et al., 2010; Vaughn & 
Wanzek, 2014). The intensity of an intervention can be 
increased by providing instruction in smaller groups, provid- 
ing longer daily lessons, providing intervention for an 
extended period of time, and modifying characteristics of the 
intervention itself, such as increasing students’ opportunities 
for engaged practice with feedback (Vaughn et al., 2010). 

In addition to their reading difficulties, students with sig- 
nificant RDs tend to have impaired self-regulation (Cutting 
et al., 2009), which can be defined as the ability to modify 
cognition, emotions, and behaviors in pursuit of goals 
(Edossa et al., 2018). Reading proficiency has been related 
to the ability to regulate one’s emotions (Daley et al., 2014) 
and thoughts (Cartwright et al., 2017). Thus, the effective- 
ness of intensive interventions for students with significant 
RDs might be increased by integrating instruction designed 
to support self-regulation. This article describes a feasibility 
study conducted to guide the development of an intensive 


intervention called Idea Detectives (ID), designed to pro- 
vide integrated instruction in reading and self-regulation for 
students in Grades 2 to 4 with significant RDs. The aim of 
this study was to collect preliminary data about the effects 
of the intervention on reading outcomes, as well as teacher 
input and feedback to guide further development activities. 


Reading Interventions for Students 
With Significant RDs 


There is converging evidence of the positive effects of 
supplemental intervention for struggling readers in the 
elementary grades (e.g., Benner et al., 2010; Wanzek et al., 
2010, 2013, 2018). Wanzek et al. (2018) conducted a 
meta-analysis of intensive reading interventions for 
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students with RDs in Grades | to 3. They concluded that 
providing standardized, explicit interventions in founda- 
tional reading skills has a positive effect for these students; 
however, they also noted a need for additional studies of 
intensive interventions for students in Grades 2 and 3. In 
addition to interventions in foundational reading skills, stu- 
dents with RDs benefit from instruction in reading compre- 
hension (Gajria et al., 2007), including instruction targeting 
inference generation (Elleman, 2017). 

Many students with significant RDs have word-reading 
difficulties (WRDs), including dyslexia. Dyslexia is charac- 
terized primarily by difficulties with reading and spelling 
words, particularly when presented out of context (i.e., in 
lists; Fletcher et al., 2019). Fletcher et al. (2019) reviewed 
research on interventions for students with WRDs, includ- 
ing dyslexia, concluding that effective interventions for 
these students include direct, explicit instruction in phonics, 
integrated with instruction in word recognition, spelling, 
reading fluency, and comprehension (Fletcher et al., 2019). 
Effective interventions for students with WRDs, including 
dyslexia, are systematic and structured, with easier skills 
taught before more complex skills (Spear-Swerling, 2019), 
and they use multiple response modalities to support learn- 
ing and retention (Fletcher et al., 2019; Spear-Swerling, 
2019). For example, word learning is supported when stu- 
dents with WRDs have the opportunity to “see the word, 
say the word, write the word, [and] use the word in text” 
(Fletcher et al., 2019, p. 190). In effective interventions, 
students have the opportunity to apply the reading skills and 
strategies they are learning while reading connected text 
with teacher feedback and support. In addition, effective 
teachers monitor students’ progress so that instructional 
adjustments can be made to address students’ needs 
(Fletcher et al., 2019). 


Self-Regulation Interventions 


Self-regulated learning requires students to actively manage 
their thoughts, emotions, and actions based on a desired out- 
come (Pintrich, 2000). Self-regulation training has been 
associated with improved use of self-regulation strategies as 
well as enhanced reading fluency and comprehension out- 
comes (Cirino et al., 2017; Miranda et al., 1997; Souvignier 
& Mokhlesgerami, 2006). 

Some types of self-regulation interventions that have 
been linked to enhanced literacy outcomes emphasized the 
following: (a) emotional self-awareness and rehearsal of 
positive self-statements, (b) beliefs about the malleability 
of intelligence, and (c) goal-setting and self-monitoring. 
When students become frustrated and lack the ability to 
self-regulate their thoughts or emotions, they can be suscep- 
tible to negative, self-defeating thoughts, leading to nega- 
tive behaviors. Emotional knowledge and awareness can 
contribute to academic performance (Gumora & Arsenio, 


2002; Trentacosta & Izard, 2007) and can be improved by 
intervention for elementary students (Kumschick et al., 
2014). Learning to recognize negative thoughts and rehearse 
positive self-statements can benefit students emotionally 
and academically (Burnett, 1999; Kamann & Wong, 1993). 
The limited research on positive self-talk during literacy 
tasks in elementary students suggests the possibility of lon- 
ger-lasting gains in reading than interventions without self- 
talk training (Berkeley et al., 2011). Students’ academic 
performance may also be linked to their beliefs about the 
nature of intelligence and academic potential. Students may 
believe that their intelligence and academic potential are 
fixed, commonly known as a fixed mind-set, or they may 
believe that those qualities are malleable and related to per- 
sistence and effort—a growth mind-set (Dweck et al., 1995; 
Yeager & Dweck, 2012). A growth mind-set is positively 
related to improved decoding and comprehension in fourth 
grade students (Petscher et al., 2017) and can be trained 
through explicit instruction (Blackwell et al., 2007) and 
purposeful feedback and praise that link effort and strategy 
use to outcomes (Mueller & Dweck, 1998). Growth mind- 
set has also been linked to self-regulatory processes, includ- 
ing goal-setting and self-monitoring (Burnette et al., 2013). 
Goal-setting interventions have led to improved reading 
comprehension in struggling fourth and fifth grade readers 
(Schunk & Rice, 1989, 1991). In addition, self-monitoring 
that requires students to track their own behavior, such as 
strategy use and progress toward goals, can improve the use 
of reading comprehension strategies and academic out- 
comes (Graves & Levin, 1989). 

Interventions combining instruction in self-regulation 
and reading comprehension have been shown to enhance 
student reading outcomes (Antoniou & Souvignier, 2007; 
Berkeley & Larsen, 2018; Dignath et al., 2008; Jitendra 
et al., 2000). Combined interventions can be more powerful 
than either self-regulation or reading instructional compo- 
nents alone (Malone & Mastropieri, 1991; Reed & Lynn, 
2016; Schiinemann et al., 2013; Spérer & Schiinemann, 
2014). This growing body of research suggests the potential 
of self-regulation training for the enhancement, generaliza- 
tion, and maintenance of reading performance. 


Purpose and Research Questions 


In this project, the research team collaborated with a group 
of special education and reading intervention teachers, 
engaging in an iterative process over a 2-year period to 
develop the ID intervention. We received input from teach- 
ers and developed portions of the intervention, and teachers 
used the materials with their students. Based on their feed- 
back and student data, we refined the intervention. This 
article describes a feasibility study conducted as part of this 
process. The study’s purpose was to obtain information 
about the feasibility of the intervention and its potential 
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effects on student reading outcomes to guide further revi- 
sions. This study was not designed to determine whether 
this intervention or similar interventions are effective. The 
research questions were as follows: 


Research Question 1: Is the intervention associated 
with stronger effects on reading outcomes than the inter- 
ventions currently provided to students with RDs in the 
participating schools? 

Research Question 2: Can teachers implement the 
intervention as designed? 

Research Question 3: What are the barriers to consis- 
tent implementation and to student progress in the 
intervention? 

Research Question 4: What are teachers’ perceptions of 
the self-regulation component of the intervention? and 
Research Question 5: What parts of the intervention 
should be maintained as they are and how should the 
intervention be revised? 


Method 


Context 


This study was conducted during the 2017-2018 academic 
year in eight schools located in two school districts in Texas. 
Both districts were historically rural but had characteristics 
of suburbs. According to the state education agency, on 
average, 31% of students who attended the participating 
schools were economically disadvantaged (SD = 23%). All 
schools met minimum state academic standards in the 
2016-2017 school year. 


Participants and Design 


The study had a quasi-experimental design. Special educa- 
tion teachers, dyslexia specialists, and reading intervention- 
ists, along with small groups of their students, were 
randomly assigned to teach the ID intervention (ID group) 
or to continue to deliver their typical school-provided read- 
ing interventions (business-as-usual [BAU] comparison 
group). However, randomization was compromised by 
changes in teachers and class configurations throughout the 
study, as described below. 


Teachers. In total, 16 teachers were originally randomly 
assigned, blocked by schools, to the ID group (NV = 9), or to 
the BAU group (N = 7); however, a total of 21 teachers 
participated in the study at some point of its implementa- 
tion. Two of the nine teachers assigned to the ID group 
withdrew from the study (with their students) after random- 
ization but before the onset of intervention because they 
were assigned to teach in a content mastery model in which 
their role was to support students with disabilities who 


received their services in a general education setting. These 
teachers were replaced by one teacher from the same school 
(with a different group of students). Three more ID group 
teachers left the study over the course of the school year: 
two left their schools and one withdrew from the study. The 
school replaced one of these with an instructional aide, who 
provided the ID intervention to one group of study partici- 
pants for the rest of the school year. To retain in the study 
the students of the other two ID teachers who left the study, 
ID was provided to these students by a certified teacher who 
was part of the research team. Thus, a total of 10 teachers 
provided intervention to students in the ID group at some 
point in the study, although only four of these were part of 
the originally randomized group. All seven teachers 
assigned to the BAU group participated throughout the 
study. In addition, four more teachers joined the study later 
in the school year when they began to teach BAU group 
students who were moved into their classrooms. This 
resulted in a total of 11 participating teachers who provided 
instruction to students in the BAU group at some point. 

The 10 teachers in the ID group were 90% female; 70% 
were Caucasian, 20% Hispanic, and 10% African American; 
60% were special education teachers, 30% were reading 
interventionists, and 10% were instructional aides. They had 
an average of 14 years of teaching experience (SD = 8 years). 
All ID teachers were certified teachers except for the one 
instructional aide who provided the ID intervention as a sub- 
stitute for a teacher who left the school. Most were certified 
to teach special education (70%) or elementary education 
(90%). The 11 teachers in the BAU group were all female; 
91% were Caucasian and 9% were African American; 27% 
were special education teachers, 55% were reading interven- 
tionists, and 18% were uncertified instructional aides. They 
had an average of 12 years of teaching experience (SD = 8 
years). In the BAU group, 36% of teachers were certified in 
special education and 82% in elementary education. 


Students. Based on teacher recommendation and a pre- 
screening word list, 89 students of the participating teachers 
were screened for eligibility using the Test of Word Reading 
Efficiency, Second Edition (TOWRE-2; Torgesen et al., 
2012). Students were included if they scored below the 25th 
percentile on either TOWRE-2 subtest. English learners 
were included if they received reading instruction in Eng- 
lish. Students with learning disabilities and other mild-to- 
moderate disabilities were included. Students were excluded 
if they had severe disabilities (e.g., severe intellectual dis- 
abilities or severe autism). In total, 48 students qualified for 
the study and had parent consent (26 ID, 22 BAU). Five left 
their schools during the study (3 ID, 2 BAU). One ID stu- 
dent was moved into a non-study class and received little ID 
intervention; this student was post-tested and retained in the 
analysis. Thus, the final analysis sample consisted of 43 stu- 
dents (23 ID, 20 BAU). 
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Table |. Student Demographic and Clinical Characteristics (Percentages). 


Characteristics Idea Detectives N = 26 


BAU comparison N = 22 Fisher Exact test p value 


Male 61.54 
Race/Ethnicity 
Caucasian 38.46 
African American 23.08 
Hispanic 23.08 
Asian 3.85 
Other 11.53 
Grade 
2 19.23 
3 53.85 
4 26.92 
Identified with dyslexia 30.77 
Limited English proficiency 11.54 
Served by special education 65.38 
Disabilities 
Learning disability 53.85 
Speech/language impairment 23.08 


Autism and related conditions 3.85 
Intellectual disability 11.54 
Other health impairment 15.38 


Note. BAU = business-as-usual comparison group. 


The average age of the students was 8.99 years in the ID 
group (SD = 0.93 years) and 9.15 years in the BAU group 
(SD = 0.82 years). Other demographic characteristics are 
found in Table 1. Although the groups did not differ signifi- 
cantly on any demographic variable, the ID group was gen- 
erally more ethnically and racially diverse than the BAU 
group. In addition, the ID group included students in Grade 
2, whereas the BAU group did not. The ID group also 
included a small proportion of students with limited English 
proficiency, whereas the BAU group did not. As for dis- 
abilities, the ID group tended to include more students with 
learning disabilities, whereas the BAU group tended to 
include more with speech and language impairments. The 
ID group included a small number of students with intel- 
lectual disabilities, whereas the BAU group did not. 


Measures 


Student measures. At pretest and posttest, word recognition, 
decoding, and reading comprehension were measured with 
the Woodcock-Johnson IV Tests of Achievement (WJIV; 
Schrank et al., 2014) Letter-Word Identification, Word 
Attack, and Passage Comprehension subtests. Letter-Word 
Identification assesses untimed word reading, whereas 
Word Attack examines the ability to read nonsense words 
as a measure of the student’s ability to decode unfamiliar 
words. In Passage Comprehension the student reads a 
phrase, sentence, or short passage and fills in missing 
words. Reliability of these subtests ranges from .89 to .96 at 


54.55 0.77 
68.18 0.05 
4.55 0.11 
18.18 0.74 
0.00 1.00 
9.09 
0.00 0.09 
59.09 
40.91 
22.73 0.75 
0.00 0.24 
72.73 0.76 
36.36 0.26 
40.91 0.22 
9.09 0.59 
0.00 0.24 
13.64 1.00 


the age range of interest (McGrew et al., 2014). The 
TOWRE-2 Sight Word Efficiency (SWE) and Phonemic 
Decoding Efficiency (PDE) subtests were administered to 
measure word reading fluency and decoding fluency. The 
child is presented a list of increasingly challenging real 
words (SWE) or nonsense words (PDE); the number from 
each list read correctly within 45 s is recorded. Test—retest 
reliability ranges from .83 to .92 (Torgesen et al., 2012). 
Oral text reading fluency was measured with the Oral Read- 
ing Fluency (ORF) subtest of the Dynamic Indicators of 
Basic Early Literacy Skills Next (DIBELS; Good et al., 
2013). The score is the number of correctly read words per 
minute. At pretest and posttest, all students in the study 
were tested using second grade DIBELS ORF passages so 
that the passages would be accessible to all students. Reli- 
ability for DIBELS ORF exceeds .80 (Goffreda & DiPerna, 
2010). 


Teacher measures. We collected an observational measure 
of fidelity and quality of implementation of the ID interven- 
tion. Lessons were video recorded and coded by two indi- 
viduals after attaining inter-rater reliability exceeding 90% 
agreement. Each instructional activity was coded on a 
3-point Likert-type scale for implementation as described in 
the teacher’s manual, provision of appropriate scaffolding 
and feedback, and accuracy of the teacher’s modeling and 
instruction. Quality ratings evaluated the teachers’ pacing 
and use of instructional time, organization of lesson materi- 
als, and students’ on-task behavior. For both fidelity and 
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Table 2. Idea Detectives Intervention Across Two Phases of Instruction. 


Component Phase | 


Phase 2 


Word study 
analysis; high-frequency word identification 

Text reading 

Comprehension 
book with post-reading discussion 


Self-regulation 
practice 


Phonemic awareness; basic letter-sound correspondences; 
decoding one- and two-syllable words; grapho-phonemic 


Reading decodable text with teacher feedback 
Pre-reading focus on a guiding question for each student 


General focus on experiencing reading success through 


More advanced sound-spelling patterns; reading 
and spelling more advanced words, including 
multisyllable words; fluent high-frequency word 
identification 

More advanced decodable text; generalization to 
non-decodable text; oral reading fluency emphasis 

Structured comprehension strategy lessons; student 
practice through oral response to student text 

Structured lessons addressing growth mind-set, 
emotional responses to reading, and self-regulated 
use of reading strategies 


quality of implementation, a score was computed for each 
observation as a percentage of the highest possible rating 
for that lesson. We set a priori criteria for “adequate” fidel- 
ity for each teacher of a mean = 80% fidelity across all of 
that teacher’s observations. The ID teachers also partici- 
pated in two focus groups to provide feedback about the 
feasibility and usability of the ID intervention. 

Teachers in both the ID and BAU groups were observed 
to compare characteristics of the interventions received by 
students in the two groups, using the Non-Evaluative 
Snapshot of Literacy Instruction (NESLI), adapted from the 
Instructional Content Emphasis Instrument-Revised 
(Edmonds & Briggs, 2003). The NESLI provides a record 
of the proportions of time in a literacy class spent address- 
ing various literacy domains, the extent of explicit teacher 
modeling or instruction and explicit student practice, time 
devoted to text reading, and the instructional materials used 
in a lesson. 


Description of the Intervention 


The ID intervention consisted of four components, address- 
ing word study, text reading, reading comprehension, and 
self-regulation. Intervention was delivered across two 
phases. In Phase 1, the primary focus was on foundational 
reading skills, whereas Phase 2 addressed more advanced 
word study and added comprehension strategy instruction 
and self-regulation lessons. Table 2 illustrates the nature of 
instruction across the phases. 


Word study. The word study component, implemented in 
both phases, provided structured, explicit instruction in 
phonemic awareness, phonemic decoding, word recogni- 
tion, and spelling. Students learned letter sounds and sound- 
spelling patterns and how to blend sounds to read words 
(1.e., a synthetic phonics approach). Unlike some phonics 
programs that teach students a variety of phonics rules and 
have students code words using the rules, in the ID inter- 
vention, students learned only a few basic rules (e.g., silent 


e rule) but practiced the recognition of phonemic and ortho- 
graphic patterns in words. Spelling instruction emphasized 
mapping sounds to print using Elkonin (1973) sound boxes 
to isolate sounds and associate them with sound-spelling 
patterns. Students received direct instruction and modeling 
from teachers, and they practiced word reading and spelling 
skills in a variety of formats, including reading, writing, and 
the manipulation of magnetic letters and letter tiles. 

The word study component was an extension of an inter- 
vention previously developed for use with first grade stu- 
dents at-risk for RDs (Solari et al., 2018). For the current 
study, we extended the first grade word study curriculum to 
address more advanced phonics skills, making it appropri- 
ate for students with RD in Grades 2 to 4. In this study, 
teachers had access to the full range of the word study pro- 
gram, including the first grade levels. Students were placed 
in the program based on a test of word-reading skills proxi- 
mal to the curriculum. Teachers monitored student progress 
in the program using mastery tests, curriculum-embedded 
assessments of student proficiency in the instructed skills. 
Teachers were provided with guidelines for using the results 
of the mastery tests to guide the pacing of instruction, some- 
times re-teaching lessons and sometimes skipping units if 
the mastery test data indicated that students had already 
mastered the skills taught in those units. 


Text reading. In both Phases | and 2, students read attrac- 
tive, motivating decodable text that followed the order of 
introduction of sound-spelling correspondences and high- 
frequency words. In Phase 2, to promote generalization, 
students also applied the reading skills and strategies they 
were learning in non-decodable text. Students learned one 
three-part strategy for reading unknown single-syllable 
words: (a) look for parts you know in the word (ortho- 
graphic patterns, letter combinations), (b) sound it out 
smoothly (without breaking between the sounds), and (c) 
check it to be sure it makes sense in the sentence. The strat- 
egy was expanded for multisyllable words to include locat- 
ing the vowels in the word as a way to identify pronounceable 
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word parts. During text reading, teachers provided prompt- 
ing and feedback to support students’ use of the reading 
strategies. Phase | text was primarily narrative, whereas 
Phase 2 included about an equal mixture of narrative and 
informational text. 


Comprehension. Phase | did not include formal comprehen- 
sion instruction, but before reading, students were given a 
guiding question to focus their reading. Questions were 
developed to support literal recall, activation of background 
knowledge, self-monitoring of meaning, question genera- 
tion, retelling (in narratives), identifying main ideas, and 
making inferences. Students and teachers discussed the 
question after reading each book. 

In Phase 2, students received structured lessons in the 
strategies of paraphrasing, identifying main ideas, visualiz- 
ing, self-monitoring, linking and integrating ideas across a 
text, and generating inferences. Instruction consisted of a 
combination of teacher modeling and student practice 
through text-based think-alouds, adapted from Denton et al. 
(2017). Teachers modeled each comprehension strategy 
through structured text-based think-alouds. Next, students 
practiced “thinking aloud” as they read, stopping after read- 
ing sections of the text to apply the strategy and talk about 
their thoughts. For example, in the first unit on paraphras- 
ing, students learned to read a section of text, close the 
book, and ask themselves, “What did I learn in this part?” 
and answer the question by telling what they learned in their 
own words (not verbatim from the text). Next, students 
learned to talk about the most important idea they learned in 
each section. In a unit on explanatory inference generation, 
students told why events happened in a narrative text or 
explained phenomena in informational texts (e.g., Why do 
groups of elephants travel from place to place?). Besides 
this text-embedded practice, students engaged in game-like 
practice activities. For example, to practice visualization, 
the teacher read a detailed description of an object (e.g., an 
empty can full of colorful wild flowers) while students lis- 
tened and created mental images of the object; then, the 
teacher showed the students three pictures of similar objects, 
and the students selected the correct one based on their 
mental images. 


Self-regulation. The ID self-regulation component consisted 
of instruction and activities designed to support a growth 
mind-set, emotional self-regulation, and self-regulated 
strategy use, and it included training in the use of positive 
self-talk, goal-setting, and self-monitoring. In initial lessons 
designed to support a growth mind-set, students learned 
about the brain and how it makes connections that can be 
strengthened through practice. They were taught that, 
although they might not be able to read proficiently yet, 
they could improve their reading through practice using the 
strategies they were learning. Because students with 


significant RDs often experience failure associated with 
reading activities, the self-regulation component taught stu- 
dents to become aware of feelings of anxiety or frustration 
they might experience while reading. Students learned 
strategies to regulate these emotional states, including posi- 
tive self-talk and a simple breathing technique. The use of 
positive self-talk was modeled by the teacher and supported 
by selected texts read aloud to the students, as well as pre- 
and post-reading activities. Finally, students learned to set 
goals related to the use of the strategies they were learning 
and to self-monitor their use of the strategies. Students 
completed a checklist during and after text reading in which 
they monitored their application of the strategies that had 
been taught in the intervention. For example, one checklist 
included the following items: (a) “Look for parts you know. 
Sound it out. Check it.” (word-reading strategy), (b) “Say it 
in your own words.” (paraphrasing strategy), (c) “Make a 
picture in your mind.” (visualization strategy), (d) “Tell 
yourself good things.” (positive self-talk), and (e) “Work 
hard! Grow your brain!” (growth mind-set). 


Implementation. Intervention was provided in small groups 
of two to four students, primarily by teachers who were regu- 
lar school district employees. ID teachers were asked to 
implement the program at least 4 days per week over 26 
weeks. ID teachers received 5 days of professional develop- 
ment across the school year, and they received ongoing on- 
site coaching from experienced master teachers who were 
part of the research team. ID teachers were observed and 
rated for fidelity and quality of implementation of the word 
study and text reading components (Phases | and 2) from one 
to three times; they were observed during the Phase 2 self- 
regulation and comprehension components from zero to two 
times. Teachers who taught the program for only part of the 
study had fewer observations. Two teachers did not teach any 
Phase 2 lessons because they left the study prior to Phase 2. 


BAU Interventions 


Overall, 86% of the students in the BAU typical school 
instruction group received supplemental reading interven- 
tions outside their regular classrooms provided by reading 
interventionists, dyslexia specialists, special educators, or 
paraprofessionals. BAU students who received supplemen- 
tal interventions received them on an average of 81 days 
(SD = 37 days), primarily in small groups, but sometimes 
individually. Ten of these students (53%), including the 
BAU students with dyslexia, received instruction using 
systematic, structured phonics programs. All of the BAU 
students with dyslexia received a structured dyslexia 
program based on a synthetic phonics approach. Six 
BAU students (32%) participated in an evidence-based, 
computerized intervention addressing word reading, fluency, 
and comprehension. Five BAU students (26%) received a 
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Table 3. Non-Evaluative Snapshot of Literacy Instruction (NESLI) Group Comparisons. 


Idea Detectives, N = 23 observations, 


BAU interventions, N = 27 observations, 


Component observed M (SD) in % M (SD) in % 
Instructional emphasis 
Word study 65 (23) 56 (36) 
Comprehension 44 (24) 35 (33) 
Vocabulary 20 (23) 28 (34) 
Fluency 23 (16) 31 (29) 
Oral language 12 (19) 4 (11) 
Writing 0 (0) 9 (15)* 
Teacher read-aloud 4 (12) 2 (6) 
Student text reading 18 (13) 21 (24) 
Explicit modeling and instruction 92 (6)** 70 (35) 
Explicit practice 94 (6)* 78 (34) 
Instructional materials 
Books 42 (19)** 21 (32) 
Student writing materials 13 (14) 65 (32)** 
Student manipulatives 13 (19) 22 (30) 
Visual aids 57 (13)* 38 (36) 


Note. Values are the proportions of lesson time that activities included each element. BAU = business-as-usual comparison group. 


p< 05. > < 01. 


fluency program incorporating repeated reading of instruc- 
tional-level text with feedback. Some BAU students received 
more than one of these interventions. 


Characteristics of the ID Intervention and BAU 
Interventions 


The results of the NESLI observations conducted in both 
the ID and BAU groups are found in Table 3. The groups 
did not differ significantly in the proportions of lesson time 
devoted to instruction in word study, comprehension, 
vocabulary, or fluency. Teachers in the BAU group pro- 
vided writing instruction for a small portion of the observed 
lessons, whereas those in the ID group did not. The groups 
did not differ in the proportion of time devoted to text read- 
ing. Explicit instruction (i.e., modeling or clear explanation, 
practice with monitoring and feedback) was observed for a 
significantly greater proportion of time in the ID lessons 
than in the BAU instruction. As for instructional materials, 
teachers in the ID group used visual aids (e.g., posters, pic- 
tures, white boards) and books in their instruction signifi- 
cantly more than did BAU teachers, whereas students in the 
BAU group used student writing materials (e.g., paper, 
worksheets, journals) significantly more than those in the 
ID group. 


Approach to Analysis 


Quantitative analysis. To examine the promise of the inter- 
vention to support student outcomes (Research Question 1), 
we conducted analyses of covariance (ANCOVAs) with 


posttest score as the dependent variable and pretest score as 
a covariate (and grade level when appropriate) to test 
whether the intervention was associated with greater effects 
on literacy outcomes than typical instruction. When possi- 
ble, we used multilevel models with child nested within 
teacher. However, due to small sample sizes, this resulted in 
a non-positive definite g-matrix for the TOWRE-2 Com- 
posite Standard Score, WJIV Word Attack, and WJIV Pas- 
sage Comprehension outcomes. The ICCs were small: 0.05 
to 0.12 for TOWRE-2 and Word Attack. For Passage 
Comprehension, the classroom variability was moderate, 
0.43, p < .05. When intervention was added as a predictor, 
the classroom variance was no longer significant, p = .09. 
Given that classroom variability was not significant, or 
dropped to nonsignificance when treatment was in the 
model, we moved to and report single-level model results 
for these outcomes. For WJIV Letter-Word Identification 
and DIBELS ORF, although teacher was included as a ran- 
dom effect, there were no significant differences among 
teachers (ps > .05). We calculated Cohen’s d effect sizes 
based on the differences between posttest means adjusted 
for pretest, with negative values indicating differences 
favoring the comparison group. 


Qualitative analysis. We conducted two focus groups with 
teachers in the ID group. Some teachers who could not 
attend were interviewed individually or provided a written 
response. Six teachers participated. Recordings of the focus 
groups and interviews were transcribed, and two research- 
ers coded each transcript and written response using NVivo 
12 Pro; all disagreements were discussed and coded by 
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Table 4. Descriptive Statistics, Group Comparisons, and Effect Sizes for the Full Sample. 


Pretest Posttest 
Idea Detectives N = 26 Comparison N = 22 Idea Detectives N = 23 Comparison N = 20 Effect size 

Score Type M (SD) M (SD) t M (SD) M (SD) f d 
Woodcock-Johnson Tests of Achievement IV: Letter-Word Identification 

WS 445.88 (20.47) 462.77 (21.03) 2.81* 457.78 (22.97) 476.35 (21.16) 0.51 -0.22 

SS 78.04 (15.05) 85.82 (15.53) 79.65 (16.38) 88.40 (15.32) 
Woodcock-Johnson Tests of Achievement IV: Word Attack 

WS 466.88 (13.99) 472.91 (13.50) 1.51 477.30 (9.35) 481.80 (12.53) 0.03 -0.05 

SS 79.77 (15.45) 84.27 (12.92) 85.61 (10.64) 88.20 (13.02) 
Woodcock-Johnson Tests of Achievement IV: Passage Comprehension 

WS 458.00 (15.68) 470.32 (12.29) 2.99" 463.30 (12.09) 475.80 (13.26) 2.12 -0.46 

SS 77.72 (13.54) 83.55 (12.44) 76.09 (11.30) 84.63 (10.61) 
Test of Word Reading Efficiency-2: Composite 

SS 70.04 (11.21) 75.73 (9.77) 1.86 73.30 (12.07) 76.60 (9.76) 0.08 0.09 
Dynamic Indicators of Basic Early Literacy SkillsK—Next: Oral Reading Fluency 

RS 47.44 (27.09) 64.16 (25.08) 2.20* 63.11 (32.48) 82.08 (24.36) 3.35 0.57 


Note. WS = W score; SS = standard score; RS = raw score (on Grade 2 text); t value = t tests to evaluate pretest differences; f value = ANCOVAs 


with pretest (and grade when appropriate) as covariates. 
*p < .05. 


consensus. Data were analyzed thematically. Themes 
addressed barriers to implementation, needed improve- 
ments of the intervention, and strengths of the intervention, 
as well as perceptions of the self-regulation intervention 
component. Utterances that included specific references to 
teacher-administered student assessments were sub-coded 
as data-based responses. 


Results 


Research Question |: Effects of the Intervention 
on Student Reading Outcomes 


Preliminary data analysis and baseline equivalence. Descrip- 
tive statistics are presented in Table 4. For WJ IV sub- 
tests, W scores were used in the analyses; however, 
standard scores are also provided for ease of interpret- 
ability and comparison to other studies. As seen in Table 
4, pretest scores on all variables were higher in the BAU 
comparison condition compared with the ID condition; 
these pretest differences were statistically significant for 
WJIV Letter-Word Identification, WJIV Passage Com- 
prehension, and DIBELS ORF (p < .05). We also tested 
whether or not groups differed by observed background 
variables at pretest. Fishers’ Exact test was used for 
binary and categorical variables as it works well with 
small sample sizes. t tests were used with continuous 
variables. There were no significant group differences on 
the child background variables of age, race, ethnicity, 
gender, EL status, or disability status. 


Group differences at posttest. Final model results and effect 
sizes are in Table 4. There were no significant group differ- 
ences for any measures at posttest. Effect sizes were negli- 
gible for WJIV Word Attack and TOWRE-2. Moderate 
effects favored the BAU comparison group for WJIV Pas- 
sage Comprehension (—0.46) and DIBELS ORF (-0.57). 
To further explore the promise of the ID intervention for 
students with dyslexia, we calculated descriptive statistics 
and effect sizes for the subgroups of students with dyslexia 
in the ID and BAU groups on four variables especially per- 
tinent to dyslexia: WJIV Letter-Word Identification and 
Word Attack, and TOWRE-2 PDE and SWE (Table 5). 
Results for students with dyslexia largely paralleled those 
for the full sample, except that, for TOWRE-2 SWE a mod- 
erate effect size (0.60) favored the ID group over the BAU 


group. 


Research Question 2: Fidelity and Quality of 
Implementation 


ID group teachers were observed to document fidelity and 
quality of implementation. Ratings are a percentage of a 
possible perfect score. The mean word study and text read- 
ing fidelity rating (across teachers and observations) was 
88% (SD = 10%), and the mean quality rating for those 
components was 92% (SD = 9%). Eight of the nine teach- 
ers who provided the word study and text reading compo- 
nents had mean fidelity ratings greater than 80% (i.e., the 
pre-established benchmark for acceptable fidelity). The 
mean comprehension and self-regulation fidelity rating was 
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Table 5. Descriptive Statistics and Effect Sizes for Students with Dyslexia. 
Pretest Posttest 
Idea Detectives N = 8 Comparison N = 5 Idea Detectives N= 8 ComparisonN=5 __ Effect size 

Score Type M (SD) M (SD) M (SD) M (SD) d 
Woodcock-Johnson Tests of Achievement IV: Letter-Word Identification 

WS 438.50 (19.40) 457.40 (17.34) 452.80 (19.75) 472.40 (20.34) -0.18 

SS 72.38 (15.68) 80.00 (12.88) 75.13 (15.07) 84.60 (14.17) 
Woodcock-Johnson Tests of Achievement IV: Word Attack 

WS 467.40 (10.93) 472.80 (14.38) 475.60 (8.09) 481.40 (10.04) —0.38 

SS 79.38 (15.39) 82.00 (14.37) 83.38 (10.03) 86.40 (10.64) 
Test of Word Reading Efficiency-2 

SWE SS 69.13 (12.11) 68.80 (7.16) 72.38 (11.70) 70.60 (9.66) 0.60 

PDE SS 68.25 (7.29) 72.20 (11.90) 74.13 (11.89) 78.00 (11.90) —0.06 


Note. Effect sizes were calculated with covariate-adjusted scores. WS = W score; SS = standard score; SWE = Sight Word Efficiency; PDE = 


Phonemic Decoding Efficiency. 


81% (SD = 11%), and the mean quality rating for that com- 
ponent was 94% (SD = 6%). Four of the seven teachers 
who provided the comprehension and self-regulation com- 
ponent had mean fidelity ratings above 80%. 


Research Question 3: Barriers to 
Implementation and Student Progress 


In focus groups and interviews, teachers identified barriers 
to implementation of the ID intervention stemming from 
school contexts (26 references). These were primarily 
related to scheduling, limited school resources, limited 
instructional and planning time, and logistics related to pro- 
viding the intervention at two different schools in the same 
day. Teachers also identified barriers to implementation and 
student progress related to student characteristics (30 refer- 
ences). These included students’ frequent frustration with 
literacy tasks, lack of confidence and inconsistent focus, as 
well as students’ serious reading impairments. Several 
teachers also cited concerns about behavior management. 


Research Question 4: Perceptions of the Self- 
Regulation Component 


Teachers were highly supportive of the self-regulation com- 
ponent of the intervention and considered it important. 
Several described their students’ need for this kind of inter- 
vention; in particular, teachers described in detail how their 
students’ negative self-concepts as readers had interfered 
with students’ reading progress. Teachers cited the positive 
effects of the emphasis on growth mind-set on students’ con- 
fidence and self-esteem. Similarly, teachers noted benefits of 
teaching students to recognize their negative self-statements 
and substitute positive self-statements. One teacher recounted 
students’ reactions when they were asked to look at a picture 


of a boy who was clearly frustrated while trying to do his 
schoolwork and talk about what the boy in the picture was 
feeling and what messages he was probably giving himself. 
The teacher recalled that it was easy for her students to pro- 
vide negative self-statements associated with this picture, 
observing, “They’ve been there!” The teacher then recounted 
feeling tearful when the students generated positive self- 
statements they could use in place of these negative state- 
ments. Two teachers noted that students “would always 
chime in with positive messages” during the lessons. Most 
teachers also had positive feedback about the self-monitoring 
activities, although some admitted they often forgot to com- 
plete the self-monitoring checklist after reading. Most teach- 
ers felt that their students were able to honestly self-monitor 
their use of reading strategies. One teacher recalled, “They . . 
. remembered the parts of the lesson ... where we did each of 
those strategies, and they were honest about what they did, so 
I do think it’s a really valuable piece . . . They’re evaluating 
themselves.” 


Research Question 5: Guidance for Revision of 
the Intervention 


Teachers provided valuable feedback for further interven- 
tion development, describing issues with the intervention 
(53 references) and providing suggestions for improvement 
(68 references). The most prominent themes were (a) chal- 
lenges related to organizing and managing the manipulatives 
and other materials, (b) parts of the program that had too 
much time devoted to teacher talk when students were 
expected to sit and listen, (c) difficulty coordinating lesson 
pacing between the word study/text reading components and 
the comprehension/self-regulation components in Phase 2, 
(d) the need for a stronger fluency component, and (e) the 
potential of incorporating technology into the intervention. 
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Some teachers also described how they had modified the 
program or its implementation to make it more feasible to 
implement and more appropriate for their students (76 
references). Some shared approaches for organizing lesson 
materials. Others stressed the importance of daily lesson 
preparation. Some teachers described how they supported 
student engagement through actions such as using student- 
generated words in activities and having students sign a 
commitment to use positive self-talk when that self-regulation 
strategy was taught. Several incorporated additional time for 
student text reading and fluency instruction. Other teachers 
adapted word study practice activities using technology 
applications available in their schools. 

In addition to their suggestions for improvement of ID, 
teachers noted many strengths of the intervention (117 ref- 
erences). Teachers’ responses reflected three main themes: 
curriculum design and content, material resources, and 
student engagement. Teachers particularly praised the 
variety of word study activities, the incorporation of non- 
decodable text in addition to decodable text in Phase 2, 
and the use of mastery tests to monitor progress in word 
study skills. Teachers were positive about the quality of 
the materials, and several described the usefulness of the 
teacher’s manuals’ scripting and scaffolding aids, espe- 
cially for new reading teachers. Teachers noted generally 
high student engagement during ID lessons, mentioning 
that their students liked the fast-paced word study activi- 
ties that often included manipulatives. Teachers said their 
students also had generally positive responses to the com- 
prehension activities. Finally, teachers noted that their stu- 
dents liked the decodable texts and that students were 
motivated by the opportunities to read challenging and 
interesting non-decodable texts with teacher support pro- 
vided when needed. 

Teachers noted several intervention benefits for their stu- 
dents (44 references), particularly in word study skills and 
strategies for decoding and spelling. Several teachers 
referred to data from the ID intervention’s mastery tests and 
additional teacher-administered tests to support their obser- 
vations of student gains. Some teachers reported that stu- 
dents were successful at generalizing strategies taught in 
the comprehension component, especially strategies for 
monitoring meaning and visualization. Regarding the self- 
regulation activities, some noted students’ ability to use 
positive self-talk and to self-monitor their use of strategies 
during reading. Several teachers also noted student gains in 
personal confidence for reading. 


Discussion 


The purpose of this feasibility study was to obtain forma- 
tive data to inform the development of an intervention to 
support reading and self-regulation for students in Grades 
2 to 4 with significant RDs, including dyslexia. Results 


indicated that the word study and text reading components 
of the intervention could be implemented feasibly by teach- 
ers in typical school settings; however, the comprehension 
and self-regulation components were more challenging to 
implement with fidelity. The intervention was not associ- 
ated with significantly better reading outcomes than other 
interventions provided to students with significant RDs in 
the participating schools. Teachers were positive about the 
intervention and cited several benefits for their students, 
and they provided valuable feedback to inform its next 
iteration. 


Student Reading Outcomes 


There were no significant differences between gains made 
by students who received the ID intervention and students 
in the BAU group, most of whom received other evidence- 
based reading interventions provided in the schools. After 
accounting for pretest and for grade level where appropri- 
ate, small to moderate effect sizes favored the BAU com- 
parison group in word reading, comprehension, and fluency. 
Thus, we do not have evidence from this study that, in its 
current iteration, the ID intervention holds promise for 
being more effective than other evidence-based interven- 
tions for supporting the reading growth of students with sig- 
nificant RDs. These results were likely related to three 
factors. First, changes are needed in the ID intervention to 
increase its feasibility and effectiveness. The qualitative 
data collected in this study will guide the revisions of the 
intervention. Second, the lack of group differences may 
have been related to the study’s small sample size and the 
fact that the BAU group had significantly higher pretest 
scores in word reading, comprehension, and fluency than 
the ID group (the same three outcomes on which effect sizes 
favored the BAU group at posttest). Although the statistical 
analyses accounted for pretest differences, the ID group 
may have included students whose severe reading difficul- 
ties were more resistant to remediation. Finally, almost 90% 
of the students in the BAU group received evidence-based 
interventions provided in small groups or individually: sev- 
eral received structured, explicit phonics programs. Thus, 
the comparison was to alternate interventions rather than to 
a “no-treatment” control group. 

When we examined the data separately for students with 
dyslexia, we found that the results largely paralleled those 
for the sample as a whole, with one difference. Although not 
statistically significant, students with dyslexia tended to 
make greater gains in word reading fluency in the ID group 
than in the BAU group. All students with dyslexia in the 
BAU group received a structured dyslexia program from 
trained dyslexia specialists. That program emphasized the 
application of phonics rules to decode unknown words, 
whereas the word study component of ID emphasized the 
recognition of phonetic and orthographic patterns in words 
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and the application of word recognition skills and strategies 
in connected text. Word reading fluency may have been bet- 
ter supported with this approach. However, the subsample of 
students with dyslexia in this study was quite small, so any 
conclusions would be premature. 

Although, on average, the ID intervention was not more 
effective than other interventions provided in the schools, ID 
teachers noted that many individual students demonstrated 
noticeable reading improvement in ID. Some ID teachers 
described specific ways their students had generalized the 
word study, comprehension, and self-regulation skills and 
strategies they had learned. Teachers frequently cited data 
from ID word study mastery tests or other teacher-adminis- 
tered assessments to support these observations. Previous 
studies of Tier 3 reading interventions have found that gains 
made by individual students varied widely, with some mak- 
ing large reading gains and others actually declining over 
time, even when provided with highly intensive interven- 
tions (Fletcher et al., 2019, p. 98). Teachers of students with 
significant RDs, including students with dyslexia, likely 
need access to more than one evidence-based reading pro- 
gram to meet the needs of all their students, who have varied 
patterns of strengths and needs and may respond differently 
to different approaches. For example, some students with 
dyslexia benefit from a rule-based approach to phonics 
instruction typical of many published dyslexia programs, 
whereas others may benefit from an approach that empha- 
sizes the recognition of sound-spelling patterns and larger 
orthographic patterns, characteristic of the ID intervention. 
Similarly, some students with dyslexia and other significant 
RDs likely benefit from comprehension and self-regulation 
instruction, whereas others may need a more exclusive focus 
on decoding, word recognition, and fluency. Future research 
should address approaches to dyslexia intervention that are 
effective for students with different characteristics. 


Teacher Perceptions, Feasibility, and Needed 
Revisions 


Teacher focus group and interview data indicated that they 
believed the intervention was feasible to implement and 
beneficial for their students, but that it could be improved. 
They considered all components of the intervention valu- 
able, particularly citing the fast-paced, hands-on word study 
activities, which kept their students actively involved, and 
the inclusion of both decodable and non-decodable text. 
Teachers also considered the self-regulation component 
helpful and important for their students. As for needed revi- 
sions, teachers stressed the need for better organizational 
strategies for the intervention materials, more active student 
involvement during comprehension instruction, and greater 
emphasis on fluency instruction. 

The fidelity of implementation data indicated that the 
word study and text reading components of ID were feasible 


for teachers to implement in authentic school settings; 
however, the comprehension and self-regulation components 
proved more challenging. This signals a need to revise these 
components. In addition, the NESLI observation comparison 
of the ID and BAU interventions revealed that students in the 
BAU interventions made greater use of writing materials 
than did the ID students. Although the comprehension com- 
ponent of ID emphasized oral response to text, it did not 
include written response to text. An increased incorporation 
of student written products may improve outcomes. 


Limitations 


Caution should be used in generalizing the findings of this 
study because it was not designed to evaluate the efficacy of 
the intervention. The study was limited by its small sample 
size and by the fact that randomization to treatment was 
compromised. Finally, the results of this study should be 
interpreted in light of the fact that most BAU students 
received evidence-based interventions provided by their 
schools, often with high intensity. 


Implications for Practice 


Teachers’ insights indicated that there is a need for an inter- 
vention like ID for students with significant RDs and that 
there is value in the incorporation of self-regulation instruc- 
tion into reading interventions for these students. However, 
this study illustrated that the development of effective and 
practical interventions for students with significant RDs is 
challenging. Too often, teachers are asked to invent interven- 
tion program themselves, utilizing a variety of materials. This 
approach likely does a disservice to both teachers and stu- 
dents. If we are to meet the goal of teaching all students to 
read adequately, the hard work of developing teacher-friendly, 
intensive literacy interventions must be prioritized. Finally, 
this study confirmed the critical importance of teacher— 
researcher collaborations in the development of educational 
interventions. Even when interventions have sound theoreti- 
cal and empirical bases, they will fail to support student out- 
comes if they are not feasible for teachers to implement. 
Engaging in an iterative development process in which teacher 
and student data inform each step of development is the only 
strategy that is likely to result in interventions that are effec- 
tive for students with dyslexia and other significant RDs. 
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